Navigation systems of the vast majority of surface, ground, and aerial vehicles has become heavily reliant on the updates received with help of the Global Positioning System (GPS). Combined with high-rate outputs from the Inertial Navigational System (INS), the integrated INS/GPS navigation system is capable of assuring accuracy within one meter for determining the vehicle's position with respect to the surface of the Earth while also contributing to a more accurate estimation of an attitude (for large vehicles). Operating on other planets or in a GPS degraded/denied environment on Earth leaves the standard INS to be the only source of the vehicle's movement estimation, and eventually causes a substantial degradation of vehicle capabilities.
There is an interest to use alternative sources to sense the vehicle's movement and rotation along with INS or even with INS/GPS, particularly with autonomous vehicles. Typically autonomous vehicles are commonly equipped with a variety of miniature passive sensors providing situational awareness. The Earth's magnetic field, position of the sun, and even the ground-sky temperature gradient are some of the environmental cues that have been leveraged for navigation purposes. In situations when a vehicle is equipped with a vision-based system, optical sensors can naturally be used as a navigation aid as well. Using computer vision to support a variety of the navigation tasks for autonomous vehicles is a rapidly growing area of development.
There is a body of literature describing different approaches to utilize simultaneous localization and mapping, especially for indoor vehicles. Such efforts may consist of pose estimation with respect to the objects of known geometry, object and obstacle detection, etc. Some approaches use an omnidirectional sensor to identify a skyline and use it for attitude and heading estimation. See e.g. Mondragón et al., “Omnidirectional Vision Applied to Unmanned Aerial Vehicles (UAVs) Attitude and Heading Estimation,” Robotics and Autonomous Systems 58(6) (2010). Others use optical sensors to navigate with respect to a moving ship, aerial fuel tankers, natural landmarks, and airports. See e.g. Yakimenko et al., “Unmanned Aircraft Navigation for Shipboard Landing using Infrared Vision,” IEEE Transactions on Aerospace and Electronic Systems 38(4) (2001); see also Valasek et al., “Vision-Based Sensor and Navigation System for Autonomous Air Refueling,” Journal of Guidance, Control, and Dynamics 28(5) (2005; see also Courbona et al., “Vision-Based Navigation of Unmanned Aerial Vehicles,” Control Engineering Practice 18(7) (2010); and see Kong et al. “Feature Based Navigation for UAVs,” IEEE/RSJ International Conference on Intelligent Robots and Systems (2006), among others. Others use image subtraction to identify and track multiple moving targets. See e.g. Jing et al., “Multi-Target Detection and Tracking from a Single Camera in Unmanned Aerial Vehicles (UAVs),” IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2016). Others use different feature detection algorithms for vehicle position estimation with the help of an image matching (IMMAT) technique. See e.g. Wessel et al., “Registration of Near Real-Time SAR Images by Image-to-Image Matching,” International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences (2007); see also Luiz et al., “Exploiting Attitude Sensing in Vision-Based Navigation for an Airship,” Journal of Robotics 2009 (2009). Image-based navigation is not possible in all weather and lighting conditions, but still offers a viable alternative and GPS backup when available.
Provided here is a method and apparatus for conducting Image-Matching (IMMAT) Navigation using location referenced aerial images in conjunction with elevation data correlated with the aerial images. The method and apparatus utilizes a library of existing aerial imagery such as satellite imagery where points on the imagery are referenced to a coordinate system, such as geographic satellite imagery referenced to a LAT/LON system, Universal Transverse Mercator (UTM) system, or some other system. The method and apparatus estimates the position of an aerial vehicle by conducting image registration between a camera image and the reference aerial images to obtain a perspective transform, then refines the estimate through comparison of points projected using the perspective transform with points generated purely through consideration of the inherent image coordinate system of the image capturing camera.
These and other objects, aspects, and advantages of the present disclosure will become better understood with reference to the accompanying description and claims.